Skip to content

Add JPEG XL (DNG 1.7 / Compression 52546) decompressor#971

Open
MaykThewessen wants to merge 5 commits into
darktable-org:developfrom
MaykThewessen:jpegxl-dng17
Open

Add JPEG XL (DNG 1.7 / Compression 52546) decompressor#971
MaykThewessen wants to merge 5 commits into
darktable-org:developfrom
MaykThewessen:jpegxl-dng17

Conversation

@MaykThewessen

Copy link
Copy Markdown

What

Adds a libjxl-backed JpegXlDecompressor so rawspeed can decode DNG 1.7 tiles compressed with JPEG XL (TIFF Compression tag 52546). It's wired into DngDecoder chunk acceptance and AbstractDngDecompressor dispatch, modelled on the existing lossy-JPEG path (JpegDecompressor). Gated behind a new WITH_JPEGXL CMake option (default ON, mirroring WITH_JPEG/WITH_ZLIB); libjxl is discovered via pkg-config.

Why

This is what Apple ProRAW uses on the 48 MP main camera of recent iPhones (16 Pro, 17 Pro): PhotometricInterpretation = LinearRaw (34892), 3 channels, 10-bit, tiled. Those files currently fail with No RAW chunks found, because compression 52546 is dropped as unsupported.

Testing

Verified by decoding real iPhone 16 Pro Max and iPhone 17 Pro ProRAW DNGs end-to-end (all tiles, 8064×6048, clean artifact-free output; 10-bit samples scale to the full 16-bit range, whitePoint 65535). Built with -DRAWSPEED_ENABLE_WERROR=ON and the WITH_JPEGXL default (no extra flags); clang-format clean.

Honest scope: this verifies decode-without-error and visually-correct output, not pixel-exact output vs a reference decoder. The 2×2 interleaved-CFA JPEG XL variant is not exercised (my samples are LinearRaw, 1×1).

Relationship to #755 / #516

There's an existing JPEG XL effort: #755 by @kmilos (and issue #516). #755 is a proof-of-concept its author noted was stalled and open for adoption, and it currently hangs on real files due to a JxlDecoderSetImageOutBuffer byte-count-vs-element-count bug (reported on #755). This PR is an independent, self-contained, CI-passing implementation offered in that spirit — not trying to compete. Happy to consolidate however the maintainers prefer: fold this into #755, co-author, or close this in favour of #755. The goal is simply to get JPEG XL ProRAW support landed. Thanks to @kmilos for the original prototype.

Scope

Only the JPEG XL piece. The other Apple ProRAW path — lossless JPEG with predictor mode 7 (iPhone ≤ 15 Pro and the telephoto cameras) — is a separate concern handled by #963.

DNG 1.7 stores the raw image with JPEG XL compression (TIFF Compression tag
52546) -- as used by Apple ProRAW on the 48MP main camera of iPhone 16 Pro and
17 Pro (PhotometricInterpretation LinearRaw, 3 channels, 10-bit, tiled). Add a
libjxl-backed JpegXlDecompressor modelled on the existing lossy-JPEG path, wired
into DngDecoder chunk acceptance and AbstractDngDecompressor dispatch.

Gated behind a new WITH_JPEGXL CMake option (default ON, mirroring WITH_JPEG /
WITH_ZLIB); libjxl is discovered via pkg-config. When disabled, the compression
is reported unsupported via a #pragma message.
@MaykThewessen MaykThewessen requested a review from LebedevRI as a code owner June 21, 2026 21:00
@dholth

dholth commented Jun 27, 2026

Copy link
Copy Markdown

toucan.zip is a sample 2×2 interleaved-CFA JPEG XL DNG compressed to be small for convenience. I took this image. Decompression should work the same whether it's lossy or lossless; then reshape to get the original Bayer pattern.
My camera produces 19M Panasonic .RW2 files, becoming 14M lossless LJPEG-92 DNGs, 12M lossless JPEG XL DNGs.
For some reason Darktable doesn't do the same dead-sensor-pixel processing when I convert to DNG compared to camera-native RW2.

I've learned that my camera is one of a few that marks its bad pixels by setting them to 0 in the RAW instead of e.g. the black threshold of 143. It is possible to translate this list to opcodes in a DNG but Adobe DNG converter doesn't. Many cameras interpolate bad pixels before writing the RAW.

For lossless I've found that the JPEG-XL modular predictor "9=leftleft" is 93% as good as 4-up for CFA data.

@MaykThewessen

MaykThewessen commented Jun 30, 2026

Copy link
Copy Markdown
Author

Follow-up verification: CFA variant + a second real file

The PR description flagged that the CFA JPEG XL variant was not exercised (my original samples were all LinearRaw 1×1). I've now closed that gap by testing against a real single-channel CFA DNG 1.7 / JPEG XL file (Panasonic DMC-GX85, toucan.lossy-d2.00-3ev.dng) alongside the iPhone 17 Pro LinearRaw file, using this branch built standalone with libjxl 0.11.2.

End-to-end decode (full decodeRaw() via rstest -c -d)

File Photometric Framing Decode Output
Panasonic GX85 Color Filter Array (1ch, 16-bit, CFAPattern2 = 2 1 1 0) bare codestream (FF 0A) OK P5 4620×3464
iPhone 17 Pro LinearRaw (3ch) ISOBMFF container (00 00 00 0C 4A 58 4C 20) OK P6 8064×6048

Both decode cleanly through the single JxlDecoderSetInput path. Worth highlighting: two JPEG XL framings occur in the wild — Apple wraps each tile in the ISOBMFF container box, while this Panasonic/Adobe-DNG-Converter file is a bare codestream. JxlDecoder auto-detects both, so the current JxlDecoderSetInput-based approach is correct, but it means a CFA-only test sample (bare codestream) and a container sample exercise different framing branches — both belong in any test corpus.

CFA layout note

This GX85 file stores the CFA as a single-channel, full-resolution mosaic (decodes to 4620×3464, 1 channel), not a 4-channel half-resolution interleaved superpixel image. So there's no de-interleave step for this variant — the decoded plane is the Bayer mosaic directly, indexed by CFAPattern2. A "2×2 interleaved" half-res encoding would be a separate representation; if a converter emits that form too, both layouts are worth covering.

Bit-depth / scaling check

The decoder requests JXL_TYPE_UINT16, which makes libjxl map the codestream's normalized [0,1] range to [0,65535]. I checked this against both files' actual codestream bit depth and DNG metadata:

File BitsPerSample codestream bits_per_sample WhiteLevel decoded white
GX85 16 12 63232 ~63232 (3952 × 16.004)
iPhone 17 10 16 65535 full-range

The key point: BitsPerSample is not the value-range indicator — WhiteLevel is. Apple declares 10-bit but stores full-range values (WhiteLevel 65535); Adobe declares 16-bit (WhiteLevel 63232). In both cases the encoder pre-scales the codestream so libjxl's [0,1]→[0,65535] mapping lands exactly on WhiteLevel. So the unconditional JXL_TYPE_UINT16 request is correct as-is — decoded values match WhiteLevel for both conventions, no decoder-side bit-depth handling needed.

(Conversely, keying the output depth off BitsPerSample would be wrong: scaling the iPhone file to its declared 10-bit would put white at ~1023 against a WhiteLevel of 65535, i.e. ~64× too dark. The current code avoids that by always producing full-range output.)

Re: the toucan dead-pixel question

For anyone following the dead-sensor-pixel thread: the GX85 DNG carries only OpcodeList3 = WarpRectilinear — no OpcodeList1, no FixBadPixels*. So the missing defect correction vs. the camera-native RW2 is not a decode issue here; the RW2→DNG converter simply didn't emit bad-pixel opcodes. (rawspeed applies OpcodeList1 Stage-1 opcodes in decodeRaw(); darktable's host side implements only OpcodeList2/3. Nothing to apply if the converter wrote none.)

Net: this branch decodes both real-world DNG 1.7 JPEG XL framings (container + bare codestream) and both channel layouts (1ch CFA + 3ch linear) correctly, with values matching the declared WhiteLevel.


Edited: corrected the bit-depth section. An earlier version of this comment suggested the decoder might need to "honor bits_per_sample" to scale output to the DNG's declared depth. That was wrong — I verified that doing so would regress the Apple files (BitsPerSample=10 but WhiteLevel=65535). The current unconditional full-range UINT16 output is correct for both Apple and Adobe DNGs.

@dholth

dholth commented Jun 30, 2026

Copy link
Copy Markdown

FYI digikam has a RW2 converter that does an excellent job copying metadata but doesn't do what I want re jxl. For me a perfect converter would either store the CFA data as lossless JXL; or interpolate out the dead pixels, arrange planes into a 2x2 layout, before doing (lossy, distance=0.5) compression.
rawspeed's RW4 record bad pixel code is here.
Where is the handling for Exif.Image.RowInterleaveFactor and Exif.Image.ColumnInterleaveFactor which are [2, 2] if each of [B G, G R] are stored in quadrants of the image?
It is also possible to use threads when decoding a single JXL tile.

@dholth

dholth commented Jun 30, 2026

Copy link
Copy Markdown

FYI Adobe's DNG SDK includes sample images, mirror here.

MaykThewessen and others added 3 commits June 30, 2026 23:30
DNG 1.7 CFA files may store the Bayer mosaic as RowInterleaveFactor x
ColumnInterleaveFactor stacked color-plane "fields" (each a smooth
half-res subimage that compresses far better under lossy JPEG XL),
signalled by tags 0xC71F / 0xCD43. rawspeed assembled the tiles but
never wove the fields back together, emitting a scrambled mosaic (four
quadrant planes stacked instead of RGGB).

Add a whole-frame de-interleave post-pass in decodeData(), run after
slices.decompress() and before handleMetadata() applies ActiveArea /
DefaultCropOrigin. It operates on the uncropped assembled buffer, one
whole pixel (getBpp() bytes) at a time, so it is type-generic over
UINT16 and F32. A single JXL tile can straddle a field boundary, so
this cannot be done per-tile.

The stored->final index math (dngDeinterleaveFieldMap) follows the
Adobe DNG 1.7.1.0 spec and the dng_sdk dng_read_image.cpp reference:
field f's rows scatter to final rows f, f+R, f+2R, ...; non-divisible
sizes let earlier fields absorb the remainder. Factored into a small
pure header (DngDeinterleave.h) so the index mapping is unit-testable.
R==1 && C==1 fast-paths to a no-op.

Also adds the missing COLUMNINTERLEAVEFACTOR (0xCD43) tag.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Unit-test dngDeinterleaveFieldMap, including the verified 4x4 R=C=2
worked example (quadrant-constant stored buffer must de-interleave to
the RGGB-phase mosaic), the per-pixel stored->final map, the factor-1
identity fast path, and the non-divisible remainder rule from dng_sdk
(total=5,factor=2 and total=7,factor=3).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The decompressor unconditionally wrote the decoded tile through
mRaw->getU16DataAsUncroppedArray2DRef() and requested JXL_TYPE_UINT16,
which asserts (dataType == RawImageType::UINT16) and fails to decode for
float (F32) JPEG XL DNGs such as Adobe's 02_jxl_linear_raw_float.dng.

Pick the JxlPixelFormat sample type from mRaw->getDataType(): request
JXL_TYPE_FLOAT and write via getF32DataAsUncroppedArray2DRef() for F32
images, keeping the JXL_TYPE_UINT16 path for integer images. The decoded
tile now lands in a type-agnostic byte buffer (libjxl reports the size in
bytes) and a templated copyTile() shares the interleaved copy loop.

Verified both 01_jxl_linear_raw_integer.dng and
02_jxl_linear_raw_float.dng decode without crashing via rstest.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@MaykThewessen

Copy link
Copy Markdown
Author

Thanks for catching that, @dholth. Let me go through your points in order, starting with a correction to my own earlier comment.


Correction: the toucan sample is 2x2 interleaved, and my earlier "OK" decode was wrong

I owe a correction to what I wrote earlier in this thread. I claimed the toucan sample (toucan.lossy-d2.00-3ev.dng) was "a single-channel full-resolution mosaic" with "no interleave factors" and "no de-interleave step for this variant", and I called the decode "successful."

That was wrong. exiftool on the file shows RowInterleaveFactor = 2 and ColumnInterleaveFactor = 2 (CFA pattern GBRG). What I was actually observing was that rawspeed decoded without crashing, which is not the same thing as decoding correctly. The decoder silently ignored the interleave tags, leaving the four half-resolution color planes stacked in quadrant layout instead of being woven back into the Bayer mosaic. The output was a scrambled mosaic. You were right.


Interleave handling: no code existed, now there is a fix (local branch, not yet pushed)

To answer your question directly: before my fix, there was no interleave handling at all. RowInterleaveFactor (0xC71F) was defined in TiffTag.h but never read; ColumnInterleaveFactor (0xCD43) was not even defined. Any 2x2-quadrant-interleaved CFA JXL DNG decoded scrambled.

I've implemented a de-interleave pass on a local branch (not pushed yet). Key design decisions:

Whole-frame post-pass, not per-tile. The de-interleave runs after all tiles are assembled, not inside the per-tile decode loop. The reason: Adobe's own canonical 2x2-interleaved test vector, 03_jxl_bayer_raw_integer.dng from the adobe-dng-sdk sample files you linked (10240x7168, 1280x1440 tiles, R=C=2), has its row-field boundary at row 3584. That is not a multiple of 1440, so a single JXL tile straddles two color planes. Per-tile de-interleave is therefore geometrically impossible; the interleave is a property of the assembled stored frame.

Index mapping. Forward map stored->final: fy = within_field_row * R + field_row_index, symmetric for columns with C. This matches the DNG 1.7.1.0 spec and the reference implementation in dng_sdk's dng_row_interleaved_image::MapRow. Final dimensions equal stored dimensions (interleave reorders, never resizes). Non-divisible field sizes are handled the dng_sdk way (earlier fields absorb the remainder row/column).

Ordering relative to crop. De-interleave runs before ActiveArea/DefaultCropOrigin is applied, matching dng_sdk's ordering.

Verified on both sample types:

File Layout Result before fix Result after fix
03_jxl_bayer_raw_integer.dng (Adobe) 1280x1440 tiles, R=C=2 scrambled 4-quadrant mosaic coherent Bayer mosaic, quadrant seam energy collapses
toucan DNG (strip-based) single strip, R=C=2 scrambled coherent
01_jxl_linear_raw_integer.dng (Adobe) LinearRaw, no interleave (unaffected) unchanged, fast-path no-op

A unit test covers the index math, including the spec's 4x4 / R=C=2 worked example and non-divisible field sizes.

I've put the fix up as a branch stacked on this PR: MaykThewessen#1 . Happy to fold it straight into this PR instead if that's cleaner for review; whatever fits how you and the maintainers want to handle it.


Bad pixels (your point 1): confirmed, nothing to fix in this PR

Your read is correct. PanasonicV4Decompressor collects zero-valued pixels into mBadPixelPositions when zero_is_bad is true (which it is for all current Panasonic cameras). The DNG path, including this PR's JpegXlDecompressor, never marks zeros, so mBadPixelPositions stays empty and fixBadPixels() is a no-op. The defect correction is lost because Adobe DNG Converter doesn't translate the camera's zero-marked bad pixels into OpcodeList1 FixBadPixels opcodes; it's a converter omission, not a decode bug in this PR. Nothing to address here.


Threading (your point 3): orthogonal to correctness

Single-tile multithreaded JXL decode via JxlResizableParallelRunner is a reasonable performance enhancement. rawspeed already parallelizes across tiles via OpenMP in the DNG decompressor, so per-tile threading would mainly benefit single-tile or few-tile images. Makes sense as a follow-up; not a correctness issue for this PR.


Separate finding: float JXL DNGs assert

While testing I hit a pre-existing issue unrelated to interleave: the current decoder writes decoded tiles through the UINT16 accessor unconditionally. Adobe sample 02_jxl_linear_raw_float.dng is a float LinearRaw DNG; it fails with an assertion. The fix needs an F32 output path: JXL_TYPE_FLOAT output format and the F32 buffer accessor when the image is float. Noting it here so it doesn't get lost, but it's a separate fix from interleave.

@github-actions

github-actions Bot commented Jul 1, 2026

Copy link
Copy Markdown

The proposed diff is not clang-formatted.
To make this check pass, download the following patch
(via browser, you must be logged-in in order for this URL to work),
(NOTE: save it into the repo checkout dir for the snippet to work)
https://github.com/darktable-org/rawspeed/actions/runs/28500053469/artifacts/8002600032
... and run:

cd <path/to/repo/checkout> # NOTE: use your own path here
unzip clang-format.patch.zip
git stash # Temporairly stash away any preexisting diff
git apply clang-format.patch # Apply the diff
git add -u # Stage changed files
git commit -m "Applying clang-format" # Commit the patch
git push
git stash pop # Unstast preexisting diff
rm clang-format.patch.zip clang-format.patch

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants